1. Field of the Invention
The invention relates to a method for the generation of a map of a scene in electronic form by picture elements (pixels), further a picture element (pixel) for an image sensor for the generation of an image with a device for exposure measurement, and an image sensor for the generation of an image in electronic form with a multiplicity of picture elements (pixels).
2. The Prior Art
Classical image processing is based on the evaluation of data delivered by an image sensor system in the form of frames. Conventional, clocked image sensors acquire the visual information from the scene either sequentially for each pixel or each pixel line/column or, in various patterns, pixel parallel, but always time-quantized at some frame rate. Each frame of data that is recorded, transmitted, and needs to be post-processed in some fashion, carries the information from all pixels, regardless of whether or not this information has changed since the last frame had been acquired, usually not long ago (e.g. Fossum E. R., “CMOS image sensors: Electronic Camera-On-A-Chip”, Electron Devices, IEEE Transactions on, Vol. 44, Iss. 10 pp. 1689-1698, October 1997). This method obviously leads, depending on the dynamic contents of the scene, to a high degree of redundancy in the image data. The problem worsens as modern image sensors advance to ever higher spatial and temporal resolution. The hardware required for post-processing of the data increases in complexity and cost, demand on transmission bandwidth and data storage capacity surges and the power consumption rises to levels that can become prohibitive, especially considering today's mobile, battery-powered applications.
For some years image sensor architectures have been studied that perform preprocessing of the visual information directly at the sensor plane, usually parallel in all pixels (“focal plane processing”). Some of these sensors send the pre-processed image information asynchronously and event-controlled, i.e. independently of external timing control (e.g. clock, shutter, reset), and only when relevant information in the scene has been detected.
In the special case of the optical transient sensor, or dynamic vision sensor (DVS), the relevant information are changes in lighting intensity received by the individual, autonomously operating pixels. The electronic circuit, “transient detector”, which is the basis for these pixels, was first reported in Lichtsteiner, P.; Delbruck, T., “A 64×64 AER logarithmic temporal derivative silicon retina,” Research in Microelectronics and Electronics, 2005 PhD, vol. 2, no., pp. 202-205, 25-28 Jul., 2005, and Lichtsteiner, P.; Posch, C.; Delbruck, T., “A 128×128 120 dB 30 mW asynchronous vision sensor that responds to relative intensity change,” Solid-State Circuits, 2006 IEEE International Conference, Digest of Technical Papers, pp. 2060-2069, Feb. 6-9, 2006, and is described in WO 2006/128315 A1.
Pixels that do not sense changes in their field of view produce no data. As a result, depending on the dynamic contents of the scene, the amount of generated data is substantially reduced as compared to conventional image sensors, which read out their entire pixel field at a constant rate, regardless of whether or not this information has changed since the last time the frame was read out.
Since changes of light intensity are usually caused by a variation in reflectance of objects in the scene, object movements are the common cause for these changes. In the data stream delivered by the sensor, only information about variable objects is contained and there are no data on homogeneous surfaces or motionless background (i.e. no conventional image data in the form of gray-level information). Since the individual pixels react asynchronously and with low latency to stimuli in their field of vision, no time quantization takes place and a high temporal resolution can be achieved. For many applications in the field of machine vision, such as automotive, surveillance, industrial automation, etc., the data delivered by a transient sensor are very well suited.
The one main restrictive feature of the described optical transient sensors is its inability to produce an intensity/grayscale image. This issue constitutes a severe limitation because many vision applications, also machine or computer vision, require information about immovable (or constant) objects or about the scene background. In addition, if human observers or operators are applied, a video picture of the regarded scene is often indispensable.
The disadvantages of conventional, clocked CMOS or CCD image sensors are, as already mentioned, the limitation of temporal resolution to the frame rate at which the pixel field is read out, the quantity of highly redundant data produced and the generally low dynamic range.
The problem to be solved by the present invention is to provide a method and an apparatus for the continuous acquisition of the full visual information of an observed dynamic scene with high temporal and intensity resolution over a wide dynamic range (of recordable and processable light intensity), and thereby generating the minimum necessary amount of data volume. Thus, the generated data are not to consist of a succession of image frames, but an asynchronous stream of change and intensity information of individual pixels, which are recorded and transmitted only if an actual change in the field of view of the individual pixel has been detected by the pixel itself. This method leads to a substantial reduction in generated data through complete suppression of the temporal redundancy in the picture information that is typical for conventional image sensors, though with the data containing the same, or even higher, information content. The picture element for an image sensor that implements the aforementioned method (being one subject of the present invention) as well as the required asynchronous data readout mechanism can be realized on basis of analog electronic circuits. An image sensor with a multiplicity of such picture elements is typically realized and fabricated as an integrated system-on-chip, e.g., in CMOS technology.